This work suggests fundamentally rethinking the current practice of pruning large language models (LLMs). The way it is done is by divide and conquer: split the model into submodels, sequentially prune them, and reconstruct predictions of the dense counterparts on small calibration data one at a time; the final model is obtained simply by putting the resulting sparse submodels together. While this approach enables pruning under memory constraints, it generates high reconstruction errors. In this work, we first present an array of reconstruction techniques that can significantly reduce this error by more than $90\%$. Unwittingly, however, we discover that minimizing reconstruction error is not always ideal and can overfit the given calibration data, resulting in rather increased language perplexity and poor performance at downstream tasks. We find out that a strategy of self-generating calibration data can mitigate this trade-off between reconstruction and generalization, suggesting new directions in the presence of both benefits and pitfalls of reconstruction for pruning LLMs.

ChatGPT, an artificial intelligence (AI) language model created by OpenAI, has been making waves across the internet, leading to questions on how AI will change the way we work and write. In the latest ICFJ Pamela Howard Forum on Global Crisis Reporting webinar, Jenna Burrell, director of research at Data & Society, dove into the pros of ChatGPT and how it can be a tool for journalists, as well as its limitations and what journalists should be cautious about. One of the most important tasks for journalists is simplifying complex topics for a general audience. ChatGPT makes this easier, Burrell said. Using the language model allows journalists to plug an abstract or part of an academic article into ChatGPT and ask the software to simplify it.

Still, it is very important to take a close look at the impact it is having on a particular task--and it is perhaps even more important to ask critical questions, as the use of AI unfolds in different vertical markets. Take for instance one very specific area today: recruitment and hiring. Roughly 95% of HR (human resources) professionals believe AI could help with the application process for candidates, according to a recent study from Tidio. With companies receiving roughly 250 resumes for each corporate job opening, AI offers the opportunity to help match the perfect candidate to the perfect job, helping minimize some of the challenges currently faced by the worker shortage. Artificial intelligence can help with screening candidates, searching for candidates on different platforms, creating job descriptions, and conducting initial interviews among other things.

Proposals would ensures that'our government understands the benefits and pitfalls of this technology as it engages in a responsible, accountable rollout of AI' A proposed US'AI in Government Act 2018' would seek to both foster bigger official support for Artificial Intelligence at the Federal level, while also highlighting areas of potential future concern. Under the Act – a bi-partisan initiative – some of Uncle Sam's executive agencies would be tasked to specifically research and consider AI applications and strategy, as well as create an advisory board to address AI policy and issues, including: Expanding the General Services Administration office to provide "technical expertise to relevant government agencies," research Federal AI policy, and "promote [American] competitiveness through agency and industry cooperation" to ensure the United Sates would have global competitive advantage in AI, supported by giving the government the resources it needs to "hire experts, do research, and work across federal agencies to use AI technologies in smart and effective ways" Establish strategies for using AI in "Federal data strategy" Identify skills and competencies for AI and establish or update what civil servants would need to know on the topic, skills-wise. The Bill's sponsors cited both the promises and risks of AI as significant motivations for their proposed legislation, noting that, "[AI] will have significant impacts for our country, economy, and society [so] ensuring that our government has the capabilities and expertise to help navigate those impacts will be important in the coming years and decades. "[This] legislation [ensures that] our government understands the benefits and pitfalls of this technology as it engages in a responsible, accountable rollout of AI." The work is the American equivalent of a Private Member's Bill, so may not progress that much – but does, in the words of The National Law Review, recognise "AI's simultaneous promises and challenges for future policymakers" and could represent "a notable step towards government adoption of AI" in America.